Add strict latest freshness validation#37
Conversation
Co-Authored-By: Codex <noreply@openai.com>
🤖 Codex PR Review🚫 Merge blocked: 2 serious issue(s) found in high-risk files 🚫 Blocking IssuesThese issues must be fixed before this PR can be merged: 1. 🟠 [HIGH] Logic in
|
Co-Authored-By: Codex <noreply@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8af26fa3ff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| - name: Validate existing latest signal if present | ||
| run: python scripts/validate_latest_signal.py --allow-missing | ||
| run: python scripts/validate_latest_signal.py --allow-missing --strict-latest |
There was a problem hiding this comment.
With the artifacts committed in this checkout, this command exits nonzero before the job can build a replacement context: data/output/latest_signal.json expired on 2026-06-30 and points at data/output/theme_momentum_snapshot.json, which is still schema 1 and has no required gate. Because this step runs first for both schedule and workflow_dispatch, the shadow dispatch workflow is blocked until a separate artifact refresh is merged; either update the committed artifacts in this change or avoid strict freshness/gate checks on the pre-existing output.
Useful? React with 👍 / 👎.
| allow_expired: bool = False, | ||
| ) -> None: | ||
| """Validate a current signal and its linked theme artifact fail-closed.""" | ||
| validate_signal(payload) |
There was a problem hiding this comment.
In strict-latest mode, a fresh schema 1 signal with no model_version or scoring_version still passes because this call intentionally preserves the historical v1 compatibility path and there is no later check that the top-level signal is schema 2. That leaves the new strict validator unable to enforce provenance/version metadata on the current latest_signal.json, even though the linked theme artifact is forced through strict v2 validation.
Useful? React with 👍 / 👎.
| "gate": { | ||
| "status": "pass", | ||
| "allow_downstream_recommendation": True, | ||
| "reasons": [], | ||
| }, |
There was a problem hiding this comment.
When the snapshot is built from partial price data, the artifact can record missing symbols, insufficient history, or unranked themes but still emits a passing gate with allow_downstream_recommendation: True. Since strict latest validation now trusts this gate as the fail-closed signal, a partial snapshot can be accepted for downstream recommendation context even when parts of the configured taxonomy had no usable ranking data.
Useful? React with 👍 / 👎.
| validate_theme_momentum_snapshot( | ||
| theme_payload, | ||
| reference_date=effective_date, | ||
| compatibility=False, | ||
| check_freshness=True, |
There was a problem hiding this comment.
This strict check only tests the linked theme artifact against the wall/reference date, so a signal with as_of January 1 can still validate while referencing a theme snapshot whose own as_of is January 20, as long as that future snapshot has not expired. That breaks the point-in-time contract for saved signals and can leak future momentum data into replay or downstream research context.
Useful? React with 👍 / 👎.
|
Superseded-by-smaller-slice: Slice B has been re-cut into B1 linkage integrity and a later B2 freshness slice. This PR is closed without cherry-picking or reusing its commits; its review/test history is retained as evidence. |
Summary
validate_signal()historical compatibility semantics unchanged--allow-expiredonly bypasses expiryValidation
python3 -m pytest -q(45 passed)python3 scripts/validate_latest_signal.py examples/latest_signal.example.jsongit diff --checkResearch-only scope; no execution, portfolio, account, or production dependency changes.